The present invention relates to a method for image-based driver identification in a motor vehicle by recording at least one two-dimensional driver facial image using a camera, temporarily storing recorded driver facial images, and comparing temporarily stored driver facial images to two-dimensional reference images of each possible driver of multiple possible drivers of the motor vehicle.
The identification of individuals on the basis of visual facial recognition is increasingly used in systems having access/usage restriction. Examples are access to security-critical areas in buildings and the use of motor vehicles. In addition, concepts are known in the motor vehicle field in particular, according to which comfort settings for a specific driver are assumed automatically after the driver has been identified using facial recognition.
For image-based facial recognition of a target person, in particular the driver of a motor vehicle, at least one image of the face is recorded by a camera. The facial image of the target person is subsequently compared by computer to stored reference images of various possible target persons, which are typically saved in a databank. This comparison may be performed on the basis of the pure image data or on the basis of features extracted from the images. The identity of the target person is subsequently judged on the basis of the degree of correspondence established upon the comparison of the images and/or their features.
In principle, it is advantageous for good recognition performance if the images compared to one another have approximately the same recording angle, i.e., the same angle between the recording camera and the facial normal lines of the target person. In addition, the majority of known facial recognition systems are based on the assumption that the essential characteristics of a face may be visually acquired best in frontal recordings.
Therefore, in most known systems, only frontal recordings are saved as reference images in a databank. As facial images of the target persons are recorded, it is also ensured by an appropriate placement of the camera and/or by an appropriate instruction to the target person that an essentially frontally recorded facial image may be generated. The placement of a camera for this purpose is connected to a high technical outlay, however. The necessity of following an instruction in regard to the head positioning and/or orientation by the target person restricts the comfort of the target person. In particular for the driver of a motor vehicle, such a method is not practical in the long-term.
Various newer methods are concerned with the object of making facial recognition more robust in relation to recording angles deviating from one another in the recorded facial image and reference image. The facial recognition can be implemented independently of the availability of a frontal facial image. Feature-based methods are generally distinguished in this case by greater robustness than methods based purely on the analysis of image data (e.g., template matching). However, the best recognition performance is also to be expected in feature-based methods upon the comparison of images having corresponding recording angles, in particular upon the comparison of frontal recordings.
A method is known from U.S. Pat. No. 6,681,032 B2, in which facial images, which are recorded from a non-frontal recording angle, are converted into a frontal image by a computing operation based on a geometrical model. The high computing outlay for the transformation, which may be implemented in an acceptable time only at high cost or not at all in a motor vehicle, is a significant disadvantage of this method. In addition, the quality of a model-based transformation of a non-frontally recorded facial image is undeniably inferior to the quality of a frontally recorded image, even in the event of high model quality.
In general, it is to be noted that methods which claimrobustness in regard to recording angles deviating from one another in the recorded facial image and reference image, such robustness is necessarily accompanied by a reduction of the reliability of a system in regard to incorrect identification. In applications in which permanent and/or repeated access of unauthorized target persons to a facial recognition system is possible under certain circumstances, such as a motor vehicle, such a reduction of the reliability is not acceptable.
Methods which calculate a three-dimensional model of the face of the target person from multiple recorded images are precluded in motor vehicles because of the high computing outlay connected with such a method. In the following, the processing of two-dimensional facial images of a target person and two-dimensional reference images is assumed.
A further approach for solving the problem of an undefined recording angle of the recorded facial image of the target person is to record multiple facial images of a target person from different recording angles and/or to save multiple reference images from different recording angles in a databank for every possible target person. The subsequent comparison of all possible combinations of a facial image to a reference image increases the probability of finding a combination having approximately corresponding recording angles.
A targeted selection of combinations having corresponding recording angles is typically not possible in such methods, because the recording angle itself is not known at least for recorded facial images. Therefore, the computing outlay required increases strongly due to the increasing combinatorial variety with increasing number of recorded facial images and/or saved reference images. Such methods are therefore unsuitable for use in a motor vehicle.
The computing outlay for the comparison operations may be reduced if a device that detects the recording angle are available in a method. Determining the recording angle from the recorded two-dimensional facial images is itself greatly subject to error, however, and requires its own computing outlay. Providing a dedicated device for detecting the recording angle required in the method for facial recognition is connected to significant technical outlay. Both solution approaches are therefore only suitable in a limited way for use in a motor vehicle.
Exemplary embodiments of the present invention provide a low-interaction and reliable method for establishing the identity of the driver of a motor vehicle by facial recognition, which may be performed without providing additional devices and with low computing outlay.
In accordance with exemplary embodiments of the present invention, the current recording angle of a device already existing in the motor vehicle, which is not used for image-based driver identification, is provided. The recording angle may therefore be stipulated as known without providing additional devices.
Furthermore, in accordance with exemplary embodiments of the present invention the recording of a driver facial image is triggered precisely when the current recording angle reaches a determined preset value. It is thus ensured that a driver facial image is recorded at a recording angle essentially corresponding to the preset value, may be temporarily stored, and may be used for comparison to reference images.
In the simplest case, a frontal recording angle is established as the single preset value, upon reaching the frontal recording angle, a driver facial image is recorded, this driver facial image is temporarily stored and used for comparison to reference images.
In accordance with exemplary embodiments of the present invention, driver facial images are exclusively compared to reference images which have essentially the same recording angle. In the above-mentioned simplest case, the frontally recorded driver facial images are therefore exclusively compared to frontally recorded reference images.
Good recognition performance is ensured in general by the comparison of driver facial images and reference images having essentially equal, ideally frontal recording angles. To further increase the reliability of a method according to the present invention for facial recognition and/or to ensure the ability to perform such a method, the following embodiments and refinements are provided.
According to various advantageous embodiments of a method according to the present invention, multiple preset values are established for recording driver facial images. Depending on the embodiment of the present invention, either all or only a part of the recorded driver facial images are subsequently temporarily stored, subsequently all or only a part of the temporarily stored driver facial images are in turn used for comparison to reference images.
Because a method according to the present invention is to be executed with as little interaction as possible, i.e., without an instruction to the operator in regard to his head positioning or orientation, it may not be predicted whether a preset value for the recording angle will actually be achieved. Therefore, driver facial images having suboptimal recording angles may be initially stored, but not use these later for comparison to reference images if, at a later time, driver facial images may be recorded and temporarily stored which may be evaluated better, i.e., are better suitable for performing the comparison to reference images at a higher reliability level. In this way, computing outlay for unnecessary comparison operations may be saved, but the possibility is retained of starting a facial recognition on the basis of the suboptimal driver facial images if needed.
In order to save additional memory space in a temporary memory for temporarily stored driver facial images, a lower number of driver facial images than were established as preset values may be temporarily stored. As a first procedure, the temporary storage or even the recording of driver facial images at suboptimal preset values may be omitted if, at an earlier time, driver facial images which may be evaluated better were able to be recorded and temporarily stored. In addition, temporarily stored suboptimal driver facial images which are already temporarily stored may be overwritten in the temporary memory by driver facial images which may be evaluated better.
Finally, a smaller number of temporarily stored driver facial images may result than the number of the preset values if the performance of comparison operations for facial recognition is begun when triggered by a specific event, such as the expiration of a timer, although a driver facial image was not yet able to be recorded for every preset value.
The sequence of overriding or replacing driver facial images temporarily stored in the temporary memory may be controlled, for example, in that preset values are each assigned a priority and temporarily stored driver facial images may be replaced by driver facial images recorded at a later time, whose preset value is assigned a higher priority.
In a comparable way, the selection of which temporarily stored driver facial images are used for comparison may be priority-controlled. For this purpose, separate priorities may be established or the priorities for the driver facial images recorded for temporary storage may be used.
Establishing the number of preset values, the preset values themselves, and their priorities, but also the establishment of the number of temporarily stored driver facial images and the number of driver facial images used for comparison may be performed at the beginning of a method and/or may be adapted to the altered situation in the course of the method, in particular after the recording or temporary storage of individual driver facial images.
Other method rules, according to which the recording, temporary storage, and analysis of driver facial images are to be performed, may be implemented in connection with the present invention.
Because a complete databank having reference images of all possible drivers of a motor vehicle from all possible recording angles may not always be assumed, the method rules to be implemented are expediently defined oriented to the distribution of the recording angles of the available reference images. For this purpose, the recording angles of the reference images must also be known and assigned thereto.
The use of multiple temporarily stored driver facial images for comparison with reference images may have various reasons as a function of the method rules used as a basis.
Firstly, the cause may be an incomplete databank of reference images. For example, the databank may not have reference images of all possible drivers of the motor vehicle for the recording angle A or the recording angle B, but at least one reference image from one of the two recording angles for every possible driver is included. In this case, a driver facial image having the recording angle A and a driver facial image having the recording angle B would preferably be used for the comparison.
In addition, to increase the reliability of a method according to the present invention, it may be advisable for multiple driver facial images from different recording angles to be compared with a reference image of the same possible driver and for the comparison results to be linked to one another. In the simplest case, a linkage may comprise selecting the comparison result having the largest or smallest correspondence. Alternatively, any arbitrary form of linkage, mathematical filtering, or statistical analysis (e.g., calculating the mean value or median) of the individual comparison results may be performed.
A method according to the present invention is based on the assumption, which is especially applicable in a motor vehicle, that a target person, the driver of the motor vehicle here, at least temporarily orients his head in a specific direction, which essentially corresponds to the travel direction, within a finite time span. A head orientation in the travel direction is already favored by the orientation of the driver seat even in the time before beginning travel, but after entering the motor vehicle. Therefore, if a camera is attached in the motor vehicle in such a way that a specific recording angle results in the event of a facial normal line of the driver oriented essentially in the travel direction, it is thus ensured that a driver facial image may be recorded at a corresponding specified preset value within a finite time. This condition is fulfilled in many modern motor vehicles by a camera belonging to a device already provided in the motor vehicle, which is not primarily used for image-based driver identification. The occurrence of all recording angles which closely surround the recording angle existing in the event of head orientation of the driver in the travel direction is also to be expected at high probability due to the natural head movements of a driver.
A further advantage of the present invention, which results therefrom, is that typically no request must be given to the driver to produce a specific recording angle by his seat position, or head and/or body posture.
In order to nonetheless be able to perform an identification of the driver in exceptional situations, according to a refinement of the present invention, after expiration of a predefined time span or upon the occurrence of a specific event, if not all driver facial images required for performing the facial recognition at a sufficient reliability level are available, a request is given to the driver to support the driver identification by assuming a specific seat position, or head and/or body posture.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.